In the case where image information with a wide band is recorded or transmitted, if the image information has its original configuration, a capacity of a recording medium and a transmitting medium is required for an amount of its data amount. Thus, efficiency of recording and transmitting is deteriorated. For this reason, an amount of data of image information is generally compressed so as to be suitable for an ability in recording of a recording medium or an ability in transmitting of a transmitting medium. Such a data compressing process is realized by encoding high-efficiently digital motion picture information by means of arts, such as motion compensation inter-frame prediction, orthogonal transformation, quantization, variable-length encoding.
For example, in a high-efficient encoding system cited in the literature "MPEG2 Interframe Prediction" (the Institute of Television Engineers, Technical Report Vol. 16 No. 61. pp. 37-42), there exists a frame on which only intra-frame encoding is performed (hereinafter, referred to as I picture), a frame on which predictive encoding from the previous frame is performed (hereinafter, referred to as P picture) and a frame on which predictive encoding from the previous and subsequent frames is performed (hereinafter, referred to as B picture). An amount of codes generated in I picture is great, an amount of codes generated in P picture is smaller than that in the I picture, and an amount of codes generated in B picture is fairly smaller than those in the I picture and the P picture.
When the high efficient encoding system is applied to image transmission and image storing (recording), there arises a problem of how a data rate after encoding (hereinafter, referred to as encoded data) is controlled.
For example, the literature "MPEG2 Quantization and Coding Control (the Institute of Television Engineers, Technical Report Vol. 16 No. 61. pp. 43-48) discloses a method for almost constantly controlling an average number of bits in a given interval which is called group of pictures (hereinafter, referred to as GOP) (an interval from I picture to next I picture, and a number of pictures in the GOP is normally ten-odd frames).
FIG. 16 is a block diagram of an image encoding apparatus utilizing a conventional method.
First, in a block dividing circuit 51, a frame to be encoded is divided into blocks which are called a macroblock (hereinafter, referred to as MB). Intra-frame encoding or inter-frame encoding is performed per MB.
In a motion compensation inter-frame predicting circuit 52, a current frame is compared with a frame previously encoded and decoded so that while an amount of a movement of MB (move vector) is being detected, the MB is predicted from a decoded frame. However, MB of I picture and MB in which intra-frame encoding is advantageous are not predicted. The predicting method is disclosed in the literature "MPEG2 Interframe Prediction" (the Institute of Television Engineers, Technical Report Vol. 16 No. 61. pp. 37-42).
In an orthogonal transforming circuit 53, the data of MB are transformed to data suitable for encoding by two-dimensional orthogonal transformation (for example, discrete Fourier transformation, Hadamard transformation, discrete cosine transformation, etc.). At this time, in the case of intra-frame encoding, two-dimensional orthogonal transformation is carried out on the data of MB from the block dividing circuit 51, whereas in the case of inter-frame encoding, a switch 54 is switched to the motion compensation inter-frame predicting circuit 52, a prediction difference (prediction error) is obtained by a subtracter 55 and two-dimensionally orthogonal transformation is carried out on the prediction error.
In a quantizing circuit 56, transform coefficients are quantized by a quantization step size obtained in a quantization step size determining circuit 66, and in a variable-length encoding circuit 57, variable-length encoding is performed on the quantized transform coefficient. Then, in a buffer 58, the output from the variable-length encoding circuit 57 is temporarily stored and encoded data are outputted at a determined constant data rate.
In addition, the transform coefficient quantized in the quantizing circuit 56 are inversely quantized in an inverse quantizing circuit 59 and inverse orthogonal transformation is carried out on inversely quantized transform coefficient in an inverse orthogonal transform circuit 60.
In the case of inter-frame encoding, since the reconstructed value from the inverse orthogonal transforming circuit 60 is the data of the prediction difference, a switch 62 is switched to a terminal 62a side, and in an adder 61, the data of the prediction difference as the reconstructed value and prediction MB outputted from the motion compensation inter-frame predicting circuit 52 are added so that reconstructed image data can be obtained. Meanwhile, in the case where the reconstructed value from the inverse orthogonal transforming circuit 60 is the data of MB on which intra-frame encoding is performed, since the reconstructed data directly becomes reconstructed image data, the switch 62 is switched to a terminal 62b side.
The reconstructed image data are stored in a frame memory 63. The reconstructed image data are used as a reference image for inter-frame prediction in the motion compensation inter-frame predicting circuit 52.
In addition, in the conventional image encoding apparatus, encoding control is performed by a bit allocating circuit 64, a reference quantization step size determining circuit 65 and a quantization step size determining circuit 66. The following will discuss the encoding control in detail.
The bit allocating circuit 64 determines a number of bits which are allocated to a frame to be encoded per frame based upon the encoded results up to the previous frame (a number of generated bits obtained from the various-length encoding circuit 57, and an average value of quantization widths obtained from the quantization step size determining circuit 66) and the given data rate. Concretely, global complexity measures X.sub.I, X.sub.P and X.sub.B of I picture, P picture and B picture are determined according to the following formulas (1) through (3). EQU X.sub.I =S.sub.I Q.sub.I ( 1) EQU X.sub.P =S.sub.P Q.sub.P ( 2) EQU X.sub.B =S.sub.B Q.sub.B ( 3)
In the formulas (1) through (3), S.sub.I, S.sub.P and S.sub.B represent an amount of encoded data of previous I picture, P picture and B picture (a number of generated bits). Moreover, Q.sub.I, Q.sub.P and Q.sub.B represent an average quantization step size at the time of encoding of previous I picture, P picture and B picture.
The global complexity measures are based upon that when an image to be encoded is complex, a number of generated bits increase and a quantization step size tends to be enlarged in order to restrict increase in the number of generated bits. However, a fixed initial value is set as the global complexity measures at the beginning of encoding.
Next, as to the bit allocating circuit 64 determines target bit allocation T.sub.I, T.sub.P and T.sub.B to an image to be encoded of I picture, P picture and B picture according to calculation of the following formulas (4) through (6). ##EQU1##
In the above formulas (4) through (6), R represents a number of remained bits in GOP, N.sub.P and N.sub.B represent a number of P pictures and B pictures which have not been encoded yet in GOP, and K.sub.P and K.sub.B represent constants.
Next, the reference quantization step size determining circuit 65 and the quantization step size determining circuit 66 determine a quantization step size by using the values T.sub.I, T.sub.P and T.sub.B. In the case of I pictures, in order to obtain a quantization step size, a reference quantization step size q.sub.j.sup.I which becomes a basis is first obtained and is compensated according to an activity of an image so that a quantization step size Q.sub.j.sup.I (j is a number of MB) is obtained. Also as to P picture and B picture, since a quantization step size can be determined in the same manner as of I picture, the following will discuss determination of a quantization step size of I picture.
First, in the reference quantization step size determining circuit 65, a reference quantization step size of orthogonal transformation coefficient is determined per MC based upon a number of generated bits up to the previous MB obtained in the variable-length encoding circuit 57 and a number of bits T.sub.I which is allocated by the bit allocating circuit 64.
In other words, the reference quantization step size determining circuit 65 determines virtual buffer fullness d.sub.j.sup.I in the j-th block of I picture according to the following formula (7). ##EQU2##
The reference quantization step size determining circuit 65 determines a reference quantization step size q.sub.j.sup.I per MC according to calculation of the following formula (8). ##EQU3##
In the formula (7), d.sub.0.sup.I is an initial buffer fullness (final buffer fullness q .sub.MBcnt .sup.I at the time of encoding previous I picture, and its initial value is a certain constant), B.sub.j is a total number of bits generated in an image up to the j-th block, and MBcnt is a total number of MB in an image. Furthermore, r in the formula (8) is a constant which is called a reaction parameter.
In the formula (7), T.sub.I bit is equally allocated to each MB, and a difference between a number of allocated bits up to the previous MB and a number of actually generated bits is reflected in the virtual buffer fullness. Therefore, if a number of generated bits up to the previous MB is great, the virtual buffer fullness becomes large, and also the reference quantization step size becomes large, so a number of subsequently generated bits is restricted. On the contrary, if the number of generated bits up to the previous MB is small, the virtual buffer fullness becomes small, and also the reference quantization step size becomes small, so a number of subsequently generated bits is enlarged. Here, in P picture and B picture, reference quantization widths q.sub.j.sup.P and q.sub.j.sup.B are determined in the same manner.
In the quantization step size determining circuit 66, an activity which shows fineness of a picture in MB obtained from the block dividing circuit 51 is calculated per MB, and the reference quantization step size q.sub.j.sup.I is changed by the activity per MB so that a quantization step size which is used in the quantizing circuit 56 is determined.
In other words, the quantization step size determining circuit 66 determines the quantization step size Q.sub.j.sup.I of the j-th MB in I picture is determined according to the following formula (9). EQU Q.sub.J.sup.I =q.sub.j.sup.I Nact.sub.j ( 9)
Here, Nact.sub.j in the formula (9) is an activity of the j-th MB which is normalized, and it is represented by the following formula (10). ##EQU4##
In the formula (10), act.sub.j is the activity of the j-th MB, and avg is an average value of the activities in a previously encoded frame (it is not always I picture) (however, a predetermined initial value is set for activity at the beginning of encoding). Here, also in P picture and B picture, quantization widths Q.sub.j.sup.P and Q.sub.j.sup.B are determined in the same manner as in the above.
In order to explain the problem of the conventional image encoding apparatus, the following will discuss a number of occupied bits in the buffer 58 in the image encoding apparatus (different from the virtual buffer fullness mentioned in the prior arts).
Here, the occupied bits in the buffer 58 is defined as follows. In other words, in the case where after encoding of each image frame and storing of encoded data in the buffer 58 are instantly completed in an image encoding apparatus, the encoded data are outputted at a constant data rate, buffer occupied bits are defined as a number of bits of the encoded data which remain in the buffer 58.
FIG. 17 shows an example of an aged change in the number of occupied bits in the buffer 58 with respect to a typical motion picture sequence (image 1). Here, a horizontal axis shows a time which is represented by a frame, a vertical axis shows the number of occupied bits in the buffer 58, and an initial number of occupied bits is 0. Moreover, a bit rate is 6 Mbps, a number of frames in GOP is 15, and an interval between I picture and P picture and an interval between P pictures are 3 frames.
As is clear from FIG. 17, at time 0 a lot of data are generated by encoding I picture, but as to subsequent P picture and B picture, an encoding amount is adjusted, so the number of occupied bits in the buffer 58 substantially returns to the initial value with respect to first images in each GOP (time 15, 30, 45, etc.). In this manner, if data amount control in the GOP is satisfactorily made, the first number of occupied bits in the buffer 58 in each GOP always returns to near the initial value. Therefore, it is sufficient that a capacity of the buffer 58 is 10.sup.6 bit, for example.
The following will discuss a motion picture sequence including stop motion or frequent scene changes. In a sequence which is shifted to a normal motion picture after stop motion continues for 30 frames (image 2), a number of occupied bits in the buffer 58 is shown by a solid line in FIG. 18. In a sequence which is shifted to a normal motion picture after frequent scene changes continue for 30 frames (image 3), a number of occupied bits in the buffer 58 is shown by a broken line in FIG. 18.
In the case where the capacity of the buffer 58 is 10.sup.6 bit, as shown in FIG. 18, data overflow greatly after time 15 and after time 30 in the image 2, that is, overflow occurs. The overflow occurs after time 15 of the image 2, for when a new GOP starts in a stop motion and I picture appears, an extraordinarily large many number of bits are allocated to the I picture and a lot of encoded data is generated. Moreover, also in I picture in a motion picture immediately after the stop motion, a lot of encoded data is generated, so overflow occurs even after time 30 in the image 2.
In order to avoid such overflow, the capacity of the buffer 58 should be enlarged, but this arises a problem that a scale of hardware increases. Moreover, in the case where a lot of encoded data is generated partially (all at once) compared to the above-mentioned normal sequence, a remaining number of bits allocated to GOP (a number of remaining bits in GOP) is extremely decreased. Therefore, there arises a problem that quality of an image thereafter is remarkably deteriorated.
In addition, there exist no encoded data in the buffer 58 after time 53 of the image 2, at times 25 through 29 of the image 3, etc., so underflow occurs. This can be a problem in actual hardware. Moreover, in the case where underflow occurs, a rate is not successively controlled, so there arises a problem that quality of an image is remarkably deteriorated due to encoding by using an inappropriate quantization step size.